Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Filebeat] Add PanOS Global Protect & User ID logs #24927

Merged
merged 9 commits into from
May 12, 2021

Conversation

legoguy1000
Copy link
Contributor

@legoguy1000 legoguy1000 commented Apr 5, 2021

What does this PR do?

Updates the Panw PanOS module to parse the Palo Alto Global Protect and User ID logs.

Why is it important?

Currently Global Protect and User ID logs are not parsed.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Author's Checklist

How to test this PR locally

Related issues

Use cases

Screenshots

Logs

@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Apr 5, 2021
@elasticmachine
Copy link
Collaborator

elasticmachine commented Apr 5, 2021

💔 Build Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: adriansr commented: /test

  • Start Time: 2021-05-12T06:49:06.511+0000

  • Duration: 113 min 34 sec

  • Commit: 657e6de

Test stats 🧪

Test Results
Failed 0
Passed 13713
Skipped 2285
Total 15998

Trends 🧪

Image of Build Times

Image of Tests

Steps errors 3

Expand to view the steps failures

filebeat-packaging-arm-arm - mage package
  • Took 9 min 43 sec . View more details on here
  • Description: mage package
x-pack/filebeat-packaging-arm-arm - mage package
  • Took 7 min 12 sec . View more details on here
  • Description: mage package
Error signal
  • Took 0 min 0 sec . View more details on here
  • Description: Error 'hudson.AbortException: script returned exit code 1'

Log output

Expand to view the last 100 lines of log output

[2021-05-12T08:40:45.394Z]  Version:           20.10.3
[2021-05-12T08:40:45.394Z]  API version:       1.41
[2021-05-12T08:40:45.394Z]  Go version:        go1.13.15
[2021-05-12T08:40:45.394Z]  Git commit:        48d30b5
[2021-05-12T08:40:45.394Z]  Built:             Fri Jan 29 14:33:34 2021
[2021-05-12T08:40:45.394Z]  OS/Arch:           linux/arm64
[2021-05-12T08:40:45.394Z]  Context:           default
[2021-05-12T08:40:45.394Z]  Experimental:      true
[2021-05-12T08:40:45.394Z] 
[2021-05-12T08:40:45.394Z] Server: Docker Engine - Community
[2021-05-12T08:40:45.394Z]  Engine:
[2021-05-12T08:40:45.394Z]   Version:          20.10.3
[2021-05-12T08:40:45.394Z]   API version:      1.41 (minimum version 1.12)
[2021-05-12T08:40:45.394Z]   Go version:       go1.13.15
[2021-05-12T08:40:45.394Z]   Git commit:       46229ca
[2021-05-12T08:40:45.394Z]   Built:            Fri Jan 29 14:31:49 2021
[2021-05-12T08:40:45.394Z]   OS/Arch:          linux/arm64
[2021-05-12T08:40:45.394Z]   Experimental:     false
[2021-05-12T08:40:45.394Z]  containerd:
[2021-05-12T08:40:45.394Z]   Version:          1.4.4
[2021-05-12T08:40:45.394Z]   GitCommit:        05f951a3781f4f2c1911b05e61c160e9c30eaa8e
[2021-05-12T08:40:45.394Z]  runc:
[2021-05-12T08:40:45.394Z]   Version:          1.0.0-rc93
[2021-05-12T08:40:45.394Z]   GitCommit:        12644e614e25b05da6fd08a38ffa0cfe1903fdec
[2021-05-12T08:40:45.394Z]  docker-init:
[2021-05-12T08:40:45.394Z]   Version:          0.19.0
[2021-05-12T08:40:45.394Z]   GitCommit:        de40ad0
[2021-05-12T08:40:45.394Z] Change ownership of all files inside the specific folder from root/root to current user/group
[2021-05-12T08:40:45.394Z] Unable to find image 'arm64v8/alpine:3' locally
[2021-05-12T08:40:46.460Z] 3: Pulling from arm64v8/alpine
[2021-05-12T08:40:46.460Z] 595b0fe564bb: Pulling fs layer
[2021-05-12T08:40:47.017Z] 595b0fe564bb: Verifying Checksum
[2021-05-12T08:40:47.017Z] 595b0fe564bb: Download complete
[2021-05-12T08:40:47.017Z] 595b0fe564bb: Pull complete
[2021-05-12T08:40:47.017Z] Digest: sha256:8f18fae117ec6e5777cc62ba78cbb3be10a8a38639ccfb949521abd95c8301a4
[2021-05-12T08:40:47.017Z] Status: Downloaded newer image for arm64v8/alpine:3
[2021-05-12T08:40:49.397Z] Change permissions with write access of all files inside the specific folder
[2021-05-12T08:40:50.671Z] Running in /var/lib/jenkins/workspace/PR-24927-17-972d6bef-a0f1-49f1-8900-ad5d28ab75a9/src/github.com/elastic/beats/build
[2021-05-12T08:40:51.532Z] + rm -rf ve
[2021-05-12T08:40:51.532Z] + find . -type d -name vendor -exec rm -r {} ;
[2021-05-12T08:40:52.770Z] + python .ci/scripts/pre_archive_test.py
[2021-05-12T08:40:53.280Z] Copy ./filebeat/build into build/filebeat/build
[2021-05-12T08:40:53.280Z] Copy ./filebeat/build/golang-crossbuild into build/filebeat/build/golang-crossbuild
[2021-05-12T08:40:53.280Z] Copy ./filebeat/build/package/filebeat-oss/filebeat-oss-linux-arm64.docker/docker-build into build/filebeat/build/package/filebeat-oss/filebeat-oss-linux-arm64.docker/docker-build
[2021-05-12T08:40:53.478Z] Running in /var/lib/jenkins/workspace/PR-24927-17-972d6bef-a0f1-49f1-8900-ad5d28ab75a9/src/github.com/elastic/beats/build
[2021-05-12T08:40:53.557Z] Recording test results
[2021-05-12T08:40:57.662Z] None of the test reports contained any result
[2021-05-12T08:40:57.669Z] [Checks API] No suitable checks publisher found.
[2021-05-12T08:40:58.538Z] + tar --version
[2021-05-12T08:40:59.797Z] + tar --exclude=test-build-artifacts-filebeat-packaging-arm-arm-tgz -czf test-build-artifacts-filebeat-packaging-arm-arm-tgz .
[2021-05-12T08:41:09.946Z] + gsutil --version
[2021-05-12T08:41:21.448Z] Masking supported pattern matches of $FILE_CREDENTIAL
[2021-05-12T08:41:22.670Z] + gcloud auth activate-service-account --key-file ****
[2021-05-12T08:41:24.354Z] Activated service account credentials for: [[email protected]]
[2021-05-12T08:41:25.479Z] + gsutil -m -q cp -a public-read test-build-artifacts-filebeat-packaging-arm-arm-tgz gs://beats-ci-temp/Beats/beats/PR-24927-17
[2021-05-12T08:41:29.207Z] + python .ci/scripts/search_system_tests.py
[2021-05-12T08:41:29.472Z] [INFO] system-tests=''. If no empty then let's create a tarball
[2021-05-12T08:41:30.368Z] + go clean -modcache
[2021-05-12T08:41:33.874Z] Cleaning up /var/lib/jenkins/workspace/PR-24927-17-972d6bef-a0f1-49f1-8900-ad5d28ab75a9
[2021-05-12T08:41:33.874Z] Client: Docker Engine - Community
[2021-05-12T08:41:33.874Z]  Version:           20.10.3
[2021-05-12T08:41:33.874Z]  API version:       1.41
[2021-05-12T08:41:33.874Z]  Go version:        go1.13.15
[2021-05-12T08:41:33.874Z]  Git commit:        48d30b5
[2021-05-12T08:41:33.874Z]  Built:             Fri Jan 29 14:33:34 2021
[2021-05-12T08:41:33.874Z]  OS/Arch:           linux/arm64
[2021-05-12T08:41:33.874Z]  Context:           default
[2021-05-12T08:41:33.874Z]  Experimental:      true
[2021-05-12T08:41:33.874Z] 
[2021-05-12T08:41:33.874Z] Server: Docker Engine - Community
[2021-05-12T08:41:33.874Z]  Engine:
[2021-05-12T08:41:33.874Z]   Version:          20.10.3
[2021-05-12T08:41:33.874Z]   API version:      1.41 (minimum version 1.12)
[2021-05-12T08:41:33.874Z]   Go version:       go1.13.15
[2021-05-12T08:41:33.874Z]   Git commit:       46229ca
[2021-05-12T08:41:33.874Z]   Built:            Fri Jan 29 14:31:49 2021
[2021-05-12T08:41:33.874Z]   OS/Arch:          linux/arm64
[2021-05-12T08:41:33.874Z]   Experimental:     false
[2021-05-12T08:41:33.874Z]  containerd:
[2021-05-12T08:41:33.874Z]   Version:          1.4.4
[2021-05-12T08:41:33.874Z]   GitCommit:        05f951a3781f4f2c1911b05e61c160e9c30eaa8e
[2021-05-12T08:41:33.874Z]  runc:
[2021-05-12T08:41:33.874Z]   Version:          1.0.0-rc93
[2021-05-12T08:41:33.874Z]   GitCommit:        12644e614e25b05da6fd08a38ffa0cfe1903fdec
[2021-05-12T08:41:33.874Z]  docker-init:
[2021-05-12T08:41:33.874Z]   Version:          0.19.0
[2021-05-12T08:41:33.874Z]   GitCommit:        de40ad0
[2021-05-12T08:41:33.874Z] Change ownership of all files inside the specific folder from root/root to current user/group
[2021-05-12T08:41:34.941Z] Change permissions with write access of all files inside the specific folder
[2021-05-12T08:41:35.151Z] Running in /var/lib/jenkins/workspace/PR-24927-17-972d6bef-a0f1-49f1-8900-ad5d28ab75a9
[2021-05-12T08:41:39.180Z] Failed in branch filebeat-packaging-arm-arm
[2021-05-12T08:41:39.244Z] Stage "Packaging-Pipeline" skipped due to earlier failure(s)
[2021-05-12T08:41:39.295Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-24927/src/github.com/elastic/beats
[2021-05-12T08:41:39.908Z] Running on Jenkins in /var/lib/jenkins/workspace/Beats_beats_PR-24927
[2021-05-12T08:41:39.954Z] [INFO] getVaultSecret: Getting secrets
[2021-05-12T08:41:39.992Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2021-05-12T08:41:40.646Z] + chmod 755 generate-build-data.sh
[2021-05-12T08:41:40.646Z] + ./generate-build-data.sh https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-24927/ https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-24927/runs/17 FAILURE 6753872
[2021-05-12T08:41:40.646Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-24927/runs/17/steps/?limit=10000 -o steps-info.json
[2021-05-12T08:41:42.508Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-24927/runs/17/tests/?status=FAILED -o tests-errors.json

💚 Flaky test report

Tests succeeded.

Expand to view the summary

Test stats 🧪

Test Results
Failed 0
Passed 13713
Skipped 2285
Total 15998

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Apr 7, 2021
@legoguy1000 legoguy1000 force-pushed the 24722/4-palo-new-logs branch 2 times, most recently from 5e23b5b to bbc2cd8 Compare April 9, 2021 15:14
@legoguy1000 legoguy1000 marked this pull request as ready for review April 9, 2021 15:15
@elasticmachine
Copy link
Collaborator

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

@mergify
Copy link
Contributor

mergify bot commented Apr 19, 2021

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b 24722/4-palo-new-logs upstream/24722/4-palo-new-logs
git merge upstream/master
git push upstream 24722/4-palo-new-logs

@mergify
Copy link
Contributor

mergify bot commented Apr 27, 2021

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b 24722/4-palo-new-logs upstream/24722/4-palo-new-logs
git merge upstream/master
git push upstream 24722/4-palo-new-logs

@legoguy1000 legoguy1000 force-pushed the 24722/4-palo-new-logs branch from c7a9127 to 902a961 Compare April 27, 2021 19:45
@mergify
Copy link
Contributor

mergify bot commented Apr 28, 2021

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b 24722/4-palo-new-logs upstream/24722/4-palo-new-logs
git merge upstream/master
git push upstream 24722/4-palo-new-logs

@legoguy1000 legoguy1000 force-pushed the 24722/4-palo-new-logs branch from 902a961 to caddfe7 Compare April 28, 2021 20:41
@mergify
Copy link
Contributor

mergify bot commented May 3, 2021

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b 24722/4-palo-new-logs upstream/24722/4-palo-new-logs
git merge upstream/master
git push upstream 24722/4-palo-new-logs

@legoguy1000 legoguy1000 force-pushed the 24722/4-palo-new-logs branch from caddfe7 to 83ebcf6 Compare May 3, 2021 14:24
@andrewkroh andrewkroh added the needs_integration_sync Changes in this PR need synced to elastic/integrations. label May 3, 2021
@mergify
Copy link
Contributor

mergify bot commented May 6, 2021

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b 24722/4-palo-new-logs upstream/24722/4-palo-new-logs
git merge upstream/master
git push upstream 24722/4-palo-new-logs

@legoguy1000 legoguy1000 force-pushed the 24722/4-palo-new-logs branch from 83ebcf6 to 9958599 Compare May 6, 2021 23:18
@mergify
Copy link
Contributor

mergify bot commented May 10, 2021

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b 24722/4-palo-new-logs upstream/24722/4-palo-new-logs
git merge upstream/master
git push upstream 24722/4-palo-new-logs

Copy link
Contributor

@adriansr adriansr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for your contribution!

Other than the conflicts and the event.outcome issue, this is looking good to me.

x-pack/filebeat/module/panw/panos/_meta/fields.yml Outdated Show resolved Hide resolved
x-pack/filebeat/module/panw/panos/ingest/pipeline.yml Outdated Show resolved Hide resolved
@legoguy1000 legoguy1000 force-pushed the 24722/4-palo-new-logs branch from 0e1c297 to 657e6de Compare May 11, 2021 22:52
@legoguy1000
Copy link
Contributor Author

@adriansr Should be GTG now.

@adriansr
Copy link
Contributor

/test

Copy link
Contributor

@adriansr adriansr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, ignoring failing arm build (known issue)

@adriansr adriansr added the backport-v7.14.0 Automated backport with mergify label May 12, 2021
@adriansr adriansr merged commit 99ba1a2 into elastic:master May 12, 2021
mergify bot pushed a commit that referenced this pull request May 12, 2021
Updates the Panw PanOS module to parse the Palo Alto Global Protect and User ID logs.

(cherry picked from commit 99ba1a2)
@legoguy1000 legoguy1000 deleted the 24722/4-palo-new-logs branch May 12, 2021 11:03
adriansr pushed a commit that referenced this pull request Jun 24, 2021
Updates the Panw PanOS module to parse the Palo Alto Global Protect and User ID logs.

(cherry picked from commit 99ba1a2)
adriansr pushed a commit that referenced this pull request Jun 27, 2021
Updates the Panw PanOS module to parse the Palo Alto Global Protect and User ID logs.

(cherry picked from commit 99ba1a2)

Co-authored-by: Alex Resnick <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
7.14 Candidate backport-v7.14.0 Automated backport with mergify enhancement needs_integration_sync Changes in this PR need synced to elastic/integrations.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Filebeat] Palo Alto integration with GlobalProtect [Filebeat] Palo Alto integration with User-ID
6 participants